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Sl'IK'l  KAI.  KSTIMATKS  OK  HAND  l,IMI  I i:i)  SIONALS 


t.  iNiuomirnoN 

A central  problem  in  the  theory  of  stationary  lime  series  is  to  estimate  the  spectrum  of 
a stationary  process  ,Y( l ) when  the  given  data  consist  of  samples  of  s(t ) f(t)  + /V(f ),  where 
/( t ) is  a detenu  in  ist  ie  trend  of  known  functional  form . In  this  paper  we  shall  consider  the 
dual  problem : the  functional  form  of  f(t ) is  unknown,  the  second-order  slat isties  of  ,Y( t 1 are 
known,  and  the  problem  is  to  estimate  the  spectrum  of  /(/ ).  The  signals  /(<  | will  he  assumed 
to  be  hand  limited  functions  having  continuous  Kourier  transforms  /(cl  which  vanish  for 
i ri  • ll  Such  signals  are  pulse-like,  and  the  amount  of  useful  information  contained  in  an 
observation  time  window  depends  as  much  on  the  position  of  the  window  as  it  does  on  its 
length.  Accurate  spectral  estimation  requires  that  the  observation  tune  window  capture  a 
significant  portion  of  the  signal  energy. 

We  shall  find  that  when  the  data  are  sampled  at  the  Nyquist  rate  21V,  consistent  spec 
tral  estimators  do  not  exist,  in  the  sense  that  infinitely  accurate  estimates  cannot  be 
obtained  from  infinitely  Ion}*  data  records.  Kor  signals  with  effective  tune  duration  /'  and 
signal  to  noise  power  ratio  (,S'/.Y),  most  of  the  useful  information  is  contained  in  a tune 
window  of  length  JVt./(21V),  where  N(.  |l/7r)y/  (2VVT1,)l't»S7N).  For  any  linear  spectral 

estimator  there  exist  signals  whose  corresponding  spectral  estimates  have  relative  mean 
square  errors  on  the  order  of  \/Nc  and  absolute  mean  square  errors  which  are  almost  as 
large  as  the  largest  produced  by  the  conventional  transform  with  ,Y(1  data  points. 

If  the  data  are  sampled  at  a rate  higher  than  211 , longer  lime  windows  can  be  effec 
lively  used,  and  the  conventional  Fourier  transform  provides  a consistent  spectral  estimator 
as  the  data  rate  increases  without  bound.  If,  however,  the  time  window  is  fixed,  consistent 
linear  spectral  est imators  do  not  exist  I lence,  to  summarize,  consistent  linear  spectral 
estimators  exist  only  if  both  the  data  rate  and  the  length  id'  the  time  window  increase 
without  bound. 

The  problem  of  spectral  estimation  is  essentially  different  from  the  problem  of  spectral 
peak  detection  and  location,  and  hence  our  pessimistic  results  concerning  the  former  do  not 
preclude  the  possibility  of  high  resolution  (supergain)  spectral  peak  detectors,  such  as  have 
been  recently  proposed  for  band  limited  signals  | l ,2,2, I | However,  improved  resolution 
necessarily  results  in  a decrease  in  accuracy  and  detectability , and  we  hope  to  discuss  this 
matter  in  subsequent  papers. 
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2.  DEFINITIONS 
2.1  The  Class  lt(W) 


Lot  //(IV)  In'  the  class  of  complex  valued  hand  limited  functions  / /(/)  whose  Fourier 

transforms  f = f(v)  are  continuous,  vanish  for  i M , and  have  / •’  derivatives  defined  almost 
everywhere.  Then 


+ w 

f(t)  j /(k)i*x|i|  2nirt  ]Jt\ 


(1) 


w 

+UO 

f(i’)  = J /(/)exp(-2jnW|(//. 


(2) 


The  functions  f of  class  / / ( W' ) are  pulse-like  in  the  sense  that  - 0 as  |t|  »°°  (a  conse- 

quence of  the  Riemann-Lebesquo  Lemma),  and  hence  this  class  does  not  contain  the 
sinusoidal  functions  whose  spectra  are  delta  functions  (line  spectra). 


We  follow  the  standard  terminology  of  radar  and  communications  theory  |5,6|  in 
defining  the  signal  energy  /•-’  = /.’(/),  mean  time  T 1(f),  and  effective  time  duration 
Te  = T,(f)  by 


+«o  + W 

j \f(t)\2dt  = J \f(v)\2dv. 


13> 


H 


+00 

r=(l/F)  | t\f{t)\~dt. 


(I) 


+vX) 


T~  = (An- IF)  j (/  M2IA/)I2<//. 

oo 

We  define  a moment  A/.,  .!/.,(/)  by 

+*-r>  +il 

M.,  = \n“  t2\f[t)\2dt  f \f'(r)r,lv 

, 'll 


(5) 


(«'») 


J 


Nlil.  ItKPOIt  I N.P!l> 


anil  wo  havo  tho  fundainontal  inequalities 

•_>  " M-jtif)  \ . I 

2 ir'-t2  | (2jtH7)-  y ' 

, 1 (r/ H')~ 

IA'')r  • H !/.,</). 

I'lioso  two  inoqiialitios  will  ho  proved  in  Soot  ion  anil  tlioy  will  ho  shown  to  ho  tho 
sharpost  possible  on  //(II  | Tho  time-bandwidth  product  211  l\  will  appear  frequently 
in  our  subsequent  discussion,  anil  wo  havo  tho  following  ‘Tinoorlainty  relation”. 


whiolt  can  ho  shown  to  ho  tho  sharpest  possible  inequality  of  this  typo  for  //( II  I 

Itcmark  1 1 ).  In  radar  and  oomnuinioations  theory,  one  lias  tho  uncertainly  relation 
|6.p.  17  1 1 

//,  /',.  ' n, 

whore  tho  effective  bandwidth  />(.  is  defined  by 

//,:  ( In--  /■')  f(r  r |-|/(C)|2(/| , 


<’  1 1 /•')  J r 

I ’sing  elementary  inequalities  one  can  easily  show  that  on  //( II  I 

i r I • H.  //.•  2nW 


2.2  The  Krror  Functions  If 2,  l{-(f),  and  It2. 

\\  hen  the  data  are  unperturbed  by  noise,  / ( i • ) will  bo  estimated  by  discrete  transforms 
of  thi1  type 


and  we  shall  derive  inequalities  of  the  type 


i/ii  i fv  I ’ '/  .(/■)< 


liOKIHIN 


" here  * (]  ■ [i  % t , ,i ) h ;i  ivri.un  explicit  function  of  the  frequency  r,  the  sample 

|Hnnt  set/  j/i.fj / \ | , ami  the  weight  s / {p|  . Pj,  .,  P\ } These  inequalities  are 

Jl ‘'r/’  1,1  ••>«*  that  for  every  given  set  of  values  for  r, /‘,p\  there  exists  a function 

/ in  //(  H ) for  which  the  inequality  (111  hecontes  an  equality.  Hence  tin1  function  Q- 
can  he  interpret  ihI  as  the  largest  possible  squared  error  in  f (i  ) (when  noise  is  absent ) 
normalized  hy  the  moment  M.,.  * 

Suppose  now  that  the  data  are  perturhed  by  at  It  lit  ivc  white  noise  ,V(  / 1 with  zero  mean 
l he  data  consist  of  samples  of  s(/|  /(/)  ♦ Y</|,  and  the  problem  is  to  estimate  fir)  by  sums 
ot  the  type  1 p„  s(/„  t.  l'his  estimate  has  mean  and  bias  equal  to  /t(rl  and  |/(r)  / (p)  | , 
respectively,  where  / 1 has  the  same  mcanmit  as  Indore  t he  variance  is  independent  of  / anil 
is  given  by 


Variance}/,  *(/„){  iri(i„;,  (12) 

"here  ii-  | \(/>  - | Kor  any  / in  //(H  i,  the  mean  square  error  in  the  spectral  estimate 
- p„  •''(/„)  is  given  by  /»’-(/')  (bias)-  ♦ variance.or 

li'if)  fir)  /,d)  ‘ ♦ u-1  ip(|  - (Id) 

ttc  define  H~  sup ! /f-(/l!  , where  the  supremum  is  taken  t>vcr  the  st’t  of  all  f m f/(W  I 
ha\ mg  a given  value  of  1/  , Then  from  the  sharpness  of  the  inequality  ( 1 1 1,  we  have 

R'f,  'K.  y‘“V.  f*.  p)  ♦ u'lifi,,  *,  (111 

and  for  every  < 0 there  exist  f 'in  //( Hi  for  w Inch  R 'if)  U ; < 


2 .1  I he  ( diiveiitional  and  l ,i|H't«l  Transforms 

f xplu  it  formulas  for  , /.  /)  for  arbitrary  values  of  r.  /’.  and  /will  be  tlerived  in 
Section  ;>  Kor  our  present  purposes  we  only  need  the  result  for  the  case  w hen  the  data 
are  ei|inspaced  and  the  weights  p art'  conventional;  i.e  , pM  r w here 

e„  -A/  exp|  2mi7M  | (Ur) 

ami  At  is  the  data  point  spacing  \Ne  write  for  (r,  /.  c’l  and  Y,  for  the  number  of 
sample  points  When  the  sample  point  set  {(„)  is  given  by  t„  n (2H  ).  where  n varies  over 
a set  of  integers  Jn  \ n \ } . then  .Y„  U.  V < 1 . and  it  (urns  out  that  is  independent 
of  i ami  is  closely  approximated  by 

y 2H’  ( i ii i 

( Ihe  at  a ur.it  y of  this  approximation  increases  with  increasing  \ and  at  \ , d it  is  within 
d of  the  true  value  1 

Kor  given  values  of  i atul  t there  exists  a unique  set  of  weights  p p(i  ./)  which 
minimizes  Q (r,  /.  p’),  ami  lienee  might  he  said  to  be  optimal  lit  the  absence  of  noise.  These 


1 


NHL  KKPORT  H322 


I 


weights  will  hereafter  be  called  the  “tapered”  weights  because  they  vanish  at  i>  = 1 W (a 
reflection  of  the  fact  that  /( 1 IV)  0 tor  all  / in  //(IV)).  The  tapered  weights  art1  denoted  by 
the  symbol  h , and  for  ease  of  notation  we  write  for  Q-  (e,  /’,  b(v,  /*)).  Formulas  for  T>  are 
given  in  Section  5.5.  Figure  1 shows  Q'f,  /Q~  as  a function  of  i>/W  for  ;VS.  = 5,  11,  21.  This 
ratio  is  an  even  function  of  i>/W  and  the  curves  are  only  plotted  for  e/lV  - 0.  Note  that 
QbIQc  = 0 at  v = i IV,  but  the  plunge  to  zero  is  very  rapid,  and  QbIQ2  0.9  over  most  of 
the  frequency  range.  Also,  the  portion  of  the  frequency  range  at  which  Q^IOc  ~ 1 increases 
as  i V,  increases. 


Fifi.  1 ~ C i'lml  vs  i'/lV 


Let  V,,,  Vb  denote  the  variances  of  the  spectral  estimates  X c„  s(tn  ) and  1'  />„  s(tn  ), 
and  let  | tn)  be  as  before.  Then  |c„|  = 1 /( 2 W ) , and  from  Hq.  (12)  we  have 

Vc=a2  NJ(4W2).  (17) 

The  ratio  Vhl V(.  turns  out  to  be  an  even  function  of  v/W , and  referring  to  Fig.  2 we  note 
the  following: 

(i)  The  plunge  to  zero  at  u = + IV  is  again  very  rapid,  and  Vh/Vc  > 1 over  most  of  the 
frequency  range. 

(ii)  The  peak  value  of  Vtt/Vc  decreases  slowly  with  increasing  Vs,  and  the  peak  loca- 
tions converge  to  v = 1 IV  as  Ns  -» °o. 

(iii)  In  Figs.  1 and  2 there  is  a close  but  not  exact  correspondence  between  the  peaks 
of  one  curve  and  the  clips  of  the  other. 
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3.  SAMIM.INC  M’TIIK  NVlJl’IST  KATK 
3.1  I’rolimmarios 

Throughout  tins  suction  it  will  always  bo  assumed  that  tbo  sample  point  sot  is  given 
by  I „ n (210,  where  tho  integer  n vanes  over  the  set  {/r  \ ' n ' V' . As  boforo.  \’s  will 

ilonoto  thi'  nunilHT  of  samplos  ( 2.Y  * 1 1.  ami  /'  will  ilonoto  tlio  l«'(ip;t li  of  tho  linio  w imlow 

Iloilo*'  .V,  ' 211  /'.  ami  tho  timo  w indow  is  | /'  2,  /'  2| 

In  partioular,  when  w o lator  oonsidor  tlio  offoots  of  expanding  tho  timo  window  . it  is 
to  bo  mulorstood  that  tho  timo  w indow  is  to  bo  prolong'd  in  both  directions.  l'lus  is 
important,  for  this  is  a nooossary  condition  for  to  converge  to  its  infimiini  as  / * 
whoro  tho  mfimimi  is  taken  ovor  tho  sot  of  oil  linear  spectral  ostimatos  1 pfls(f„  1 l’ho  proof 
of  this  statomont  is  todious  and  will  not  bo  given;  however.  it  is  easy  to  soo  why  it  must  bo 
truo  booauso  of  tho  pulse  liko  nature  of  tho  functions  of  class  //(H  I. 


b 


.1 


\UI  KKI'Olil  > 


;i.-J  Hi,-  Parameter  N(. 

\\e  now  assume  that  noise  is  present  ami  that  tin*  conventional  Fourier  transform  is 
used  < „ 1 t hen  from  Fps  (1H  and  (17)  ami  the  approximation  ( It'd  we  have 

dll  \/  .,  <r’\ 

K’  ' ♦ tl>s' 

ir  \ t»  ' 

> 


I lenee  />'  ' is  a eomex  funetion  of  \ v . ami  its  minimum  value  is  a (aim'd  at  \ , 

811  ; 1/ 

\ 

n*o" 

For  functions  / satisfying  /(  ' 0.  l ap  (fit  y telds  V..  /'^  f , and  we  have 


\ . vv  here 


\ (I  “I 


('.•it  r.F’tN  M 


w here  the  signal  to  noise  power  ratio  is  defined  hy 


time  averae.ed  signal  power 
(8  VI  . or 

av erase  noise  pow  er 


f r 


t-s  N > 


tl'.'t 


(L’tM 


tan 


the  nuinher  V has  another  interestim;  property  When  t is  an  integral  multiple  ol 
1 (211  ),  from  Fat  (71  we  have 


. H >1., 

MM  - 

2a- r 

aml  lienee  any  f in  ll[\\  t eventually  Ixvomev  luiriol  m noise  le  there  exists  a numher  / , 
sm  h that 


M 1 1 e when  ! ' F , 

lveealluii:  that  the  data  time  vv  indow  is  | F F d|,it  is  easy  to  verify  that  the  dominion 
\ is  eqtuv alent  to  the  eomht ion  F F 

Mthoiifjli  IF,  I'oi  -nines  an  im  reasim:  funetion  of  \,  when  \s  \ . we  eannot  assume 
that  tin-  same  is  true  for  K (M  for  any  purfi,  However,  it  ean  he  shown  that  for  any 

ill  //(((  t.  K (M  eventually  heeomes  an  m«  reasim:  funetion  of  Y . and  usuii;  some  cross 
estimates  this  ean  he  shown  to  he  the  ease  when  \ to  prove  this  result,  we  eonsider 

tile  variation  of  the  rii’.ht  hand  side  of  Ftp  ( Id)  when  the  value  ot  is  meroasod  hy  unity 
l'lie  first  term  Mi  l /'  tel  t an  eham;e  hy  mi  im'iv  than  its  largest  ptissihh'  value,  vi  . 


(iOKlM)N' 


)/(n~Ns)-  Also,  tilt*  second  term  changes  (exactly)  by  the  amount 

d*/(4H'-).  Hence  the  not  change  must  lx-  positive  when  A ■ HW'M.Jix-n-)  \~ 

4 - 1 ' (' 


Nonexistence  ot  Consistent  estimators 

In  the  last  paragraph  we  saw  that  when  the  eonventional  Fourier  transform  is  used 
the  error  function  R~  eventually  becomes  an  increasing  function  of  ,Vv  as  N • ■ »-  For  each 
value  of  Av  we  shall  now  choose  a set  of  weights  p’  which  is  "optimal"  in  the  sense  that  it 
minimizes  the  right  hand  side  of  Kq.  (11)  forgiven  values  of  M.,  and  o-\  these  weights  will 
lx’  i ailed  o-optimal,  and  when  they  are  used,  R~  becomes  a monotonicalty  decreasing 
function  of  Ns.  We  define 


R'z>  = lint  R~  (22) 

.V  »°° 

$ 

and  from  the  definitions  it  is  evident  that 

N'io  inf  J (2d) 

w here  the  infimum  is  taken  over  the  set  of  all  linear  spectral  estimates,  1 lence.  for  any 
linear  spectral  estimate  1'  p ) and  for  any  ^ 0 there  exist  functions  / in  //( ll) 
satisfying 


'■  R'L  iS. 

1 he  calculation  of  ~ , will  be  described  in  Section  f>. 5 It  turns  out  that 


in/., 


cos ft-  |/r.V(  2|  cos /r  |tr.Y.r  (211 

( n.V  2)  sin/i  t.\ 


(•21) 


ll  follows  that  ■ (),  except  when  r 1 11  Hence,  consistent  spectral  estimators  do  not 
exist  when  the  noise  is  white  and  the  data  are  sampled  at  the  N'yquist  rate 


2.1  Conclusions 


We  shall  now  consider  some  consequences  of  Kq.  (21)  For  notational  ease  we  set 


Then 


it  rrX  2,  <»  r )l 
( 


cos h~ u cos lr 


ti  sin/i  2 it 


(25) 


( 2b ) 


S 


ll  '/., 
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Let  .Yt.  denote  the  value  of  R~  when  the  conventional  transform  is  used  and  Ns  = Nc . From 
F.qs.  ( 18)  and  ( 19)  we  get 


.V  = 
c 


2A/.,o 


2 


Wtt'2 


(27) 


and  therefore 


nfjx,.  - u/2) 


cos u - cos h’  (xu 
sin/i  2 u 


(28) 


When  e is  close  to  * W it  is  not  unreasonable  to  compare  Ii with  the  “trivial”  estimate 
f(v)  0.  1’his  estimate  has  no  variance,  and  its  mean  square  error  XQ  is  given  by  the  right- 
hand  side  of  the  (sharp)  inequality  (8).  Hence, 


KtoIX, 


1-0“ 


cos/i 2 u - cos h2  Ml 
u sin h 2 u 


(29) 


The  right-hand  side  of  Kq.  (28)  attains  its  largest  value  when  a = 0,  where  it  equals 
(7r/  l)tanh  u,  and  referring  to  Fqs.  (9)  and  (20)  one  can  easily  verify  that  tanh  u ^ 1 when 
f.S'/JV)  > 1. 


The  ratio  R^0IXC  decreases  monotonically  as  u increases  (i.e.  the  conventional  Fourier 
transform  improves  as  Ns  increases).  When  i>  = 0.9  and  .\'r  - 1 , the  ratio  is  0.21.  This  means 
that  for  any  linear  spectral  estimate  there  exist  functions  f in  H(W ) for  which  R‘*(f)  exceeds 
one-fifth  the  largest  mean-square  error  produced  by  a conventional  one-point  transform 
(which  occurs  at  some  function  different  from  f). 


We  now  examine  the  right-hand  side  of  Kq.  (29).  This  ratio  converges  to  unity  as 
v • 1 IT,  and  its  minimum  value  is  attained  at  i>  = 0,  where  it  equals  (1  /u)  tan h u - i/u  = 
2RtiNc).  We  therefore  draw  the  following  conclusions: 


Conclusion  I. 


When  the  data  are  sampled  at  the  Nyquist  rate,  every  linear  spectral  estimator  produces 
mean  square  errors  having  the  same  order  of  magnitude  as  the  lari;cst  produced  by  a conven- 
tional Fourier  transform  with  Nc  data  points,  except  near  v - t IT,  where  the  errors  have  the 
same  order  of  magnitude  as  those  produced  by  the  trivial  estimate  fU>)  0. 

The  ratio  R^0/Xo  can  also  be  interpreted  as  a bound  on  the  relative  errors  R~(f)/\f(i>)'r . 
For  there  exist  f for  which  R2(f)  > R%o,  and  for  any  f we  have  i ' .Y„  since  .Y()  is  the 
right-hand  side  of  Kq.  (8).  Hence,  there  are  always  f for  which  /i2(/')/|^(p)i2  exceeds  R%J. Y,,, 
and  we  are  led  to  the  following: 
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Conclusion  2. 


When  the  data  arc  sampled  at  the  Nyquist  rate,  every 
relative  mean  square  errors  which  are  on  the  order  of  l/Nc 


linear  spectral  estimator  produces 
near  v = 0 and  unity  near  v = t H'. 


1.  OVERSAMPLING 


•I.  1 The  Case  of  Variable  T 


In  this  section  we  consider  the  effects  of  oversampling.  That  is,  we  now  suppose  that 
functions  of  class  //( H ) arc  sampled  at  a rale  2k  W,  k>\.  As  before,  we  assume  that  the  time 
window  is  of  the  form  [-77 2,  772| , so  that  the  window  is  prolonged  in  both  directions  when 
/ is  increased.  We  also  assume  that  the  noise  remains  white,  which  is  usually  the  case  for 
receiver  and  thermal  noise. 

1 he  results  of  Section  3 are  based  on  certain  closed-form  expressions  for  Qft  and  /(-. 
which  are  derived  in  Section  5 for  the  case  of  sampling  at  the  Nyquist  rate.  These  deriva^ 
tions  require  the  closed-form  inversion  of  certain  matrices,  which,  unfortunately,  wc  have 
been  unable  to  effect  for  the  case  of  oversampling.  Hence  we  are  presently  unable  to  give  a 
quantitative  description  of  how  much  useful  information  is  contained  in  a fixed  time  window 
when  the  data  rate  is  increased  without  bound.  However,  it  can  be  shown  that  consistent 
spectral  estimators  exist  only  if  both  the  data  rate  2k  W amt  the  length  T of  the  time  window- 
are  allowed  to  increase  without  bound.  Results  of  this  type  are  apparently  known,  and  our 
discussion  will  be  brief.  (Cf.  the  discussion  in  Section  B.l  of  Blackman  and  Tukey  [7]  which 
suggests  the  existence  of  results  of  this  nature.) 

I he  t lass  //(  W ) can  be  considered  as  a subset  of  It(hW),  and  error  bounds  can  be 
obtained  by  replacing  each  occurrence  of  W with  kW  in  the  formulas  of  Section  3.  The 
error  bounds  thus  obtained  are  sharp  for  the  class  ll{kW),  but  are  not  sharp  for  the  class 
//( W),  and  we  shall  refer  to  this  method  as  the  “inexact”  method. 

Recall  that  the  error  function  R~  of  the  conventional  transform  attains  its  minimum 
value  Xc  when  Ns  = Nc , or  equivalently,  when  T = NC/(2W).  One  effect  of  increasing  the 
data  rate  is  to  increase'  the  value  of  Nc  and  hence  permit  the  use  of  longer  data  records.  For 
by  applying  the  inexact  method  to  Eq.  ( 19),  we  see  that  Nc  increases  by  a factor  of 
when  W is  replaced  with  kW . Moreover,  from  Eq.  (27),  Xc  decreases  by  the  factor  \j\fk. 
Hence,  conventional  transform  provides  a consistent  spectral  estimator  if  the  length  of  the 
time  window  is  increased  by  a factor  of  y/k  when  the  sampled  rate  is  increased  to  21;  H 
and  h -»  <». 


1.2  The  Case  of  Fixed  '!’ 

U e shall  now  suppose  that  T is  fixed,  and  show  that  consistent  spectral  estimators 
do  not  exist  when  k • «>.  The  proof  uses  Shannon’s  Sampling  Theorem,  according  to 
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which  every  hand  limited  function  f whose  Fourier  transform  is  of  class  /.L>  has  the 
representation 


loo 

sin  |fr(n  2W't)| 

" |tr(«  2HT)|  ’ 

n=-°° 

where  we  set  fn  f\n/{2W)] . The  hi  infinite  sequence{/„ ) belongs  to  the  class  i.e., 


+oo 


LX'  <' 


CM) 


For  each  h "■  1 the  data  s(t)  «(/)  + N(l)  are  sampled  in  the  window  | 7/2,  772 1 at 
the  points  t n/(2kW),  where  the  integers  n are  hounded  by 


Ini  ■ /Vfc  integral  part  of  (kWT). 

I.<,tp,,‘t  \pjlli'i,(  V k ' fl  -Vfc  ) he  a weight  vector  which  produces  the  spectral  estimate 
— fill'  I '•/(  2k  W ) | We  shall  suppose  that  the  fi^' ' can  he  chosen  to  make  (lit1  mean  square 
error  converge  to  zero  as  k • and  derive  a contradiction.  Setting  I n/{2kW)  m Kq  (30) 
we  get  the  formal  result 


" 1 Nk  m Nh  in  Nk 


Using  Kq.  (.'!()),  the  first  sum  of  the  right-hand  side  can  In-  expressed  as  an  infinite  series 
involving  the  functional  values  /'„  /'|«/(2H')| , and  this  series  can  easily  he  shown  to 

converge  if  the  weight  vectors  (Ilk)  an>  chosen  to  make  the  mean  error  converge  to  zero 
as  /.•  ■ I lie  second  sum  can  he  expressed  as  a finite  linear  combination  of  the  values 
iVU  >|  f/(2H  ) | . where  the  noise  process  N^' '( t)  N[l Ik)  is  assumed  to  he  white  Hence  every 
estimator  of  type  (HI ) is  a linear  combination  of  functional  values  of  / and  white  noise 
spaced  at  At  I (211  ),  and  therefore,  from  the  results  of  the  previous  section,  cannot  lie 
consistent 


r>.  matiikmaticai.  m im  atioms 


h.  1 ( ieneral  Theory 


We  first  consider  I lie  noise-free  case,  and  describe  (he 
calculation  of  the  error  function  (}  '.  Hefcrring  to  Kq.  ( 10) 
to  define  the  function  Q~  Q2(v,  i,  /!)  by 


mathematical  tools  used  in  the 
for  notation,  it  is  now  convenient 


Q- 


si  IJ 

f 


WU'l  /'*('  ) ’I 

I 11m2  \ 


U12) 


I I 


lion  IH»N 


whi'D1  11*11  is  .1  i w win  ili'fmi'il  i mi  it  1 1 illii’rl  twiii'liun  span  f //,  and  tin*  su  pro  mum  ts  t akf  n 
ovrr  ;ill  / lif  longing  to  II  1 li'inr 

!A*-»  l\o  ) - ii/ii ' f",  *{).  t;t:n 

iii'il  • 'in  IliUiiWt  spaff  gfiii-ralitifs  this  ini-quality  is  sharp.  UV  shall  sul'sfqtifiitly  takf 
II  1 1(  ll  I aiul  / M i\  / i.  so  1 1 1 at  tin-  sharpness  ot  Kq .( implies  I hat  Ki  | (111^ 

•sharp,  hut  ivr  shall  lust  dismiss  t lit'  theory  m more  general  tonus. 

In  i hi liw  lor  tlif  rinlit  hand  sulf  of  Kq.  |.il2)  to  In-  finiti*.  it  is  neeessary  that  tin-  norm 
11*  satisfy  the  following  two  eondit ions 

I'nmhtion  I Kur  eaeli  /,  tin-  rvaluation  map  / * /(f ) is  continuous  with  respeet  to  li* 

and 

('omtitum  2 Kor  eaeli  e,  the  map  / * /(i  ) is  eontmuous. 

ililhfit  finu  tion  spates  II  satisfying  Condition  I are  failed  reproducing  kfriifl  llillu-rt 
sl >;u*«'s  |S|,and  lor  siifh  spai  fs  tin-  '.lies/  Keprescnlalion  I'hforfiti  guarantees  t lit*  existence 
ot  fuiift  iiiiis  K , A ,fst  satisfying 


/(/)  (/.  K, ),  for  all  / in //,  (dl) 

w hfif  ( * ,*  I dfiiotfs  tlu'  ituifV  )>roitiu*t  corresponding  to  tlif  norm  |p  II  The  I lorni it  inn 
matrix 


H,is)  t Kf,Ks)  (.15) 

is  called  tlif  reproducing  kernel  Sim ilarlx  . wlifii  Condition  2 is  satisfied  there  exist 
Millet  ions  i',.  «•,.(/ 1 in  II  dc fined  hy 


t/.  >',)■ 

U<‘  fan  now  w rite  I'q  td'Jt  m tlif  form 

■ ( U\>\  h'J  I 

Q sup  »•  V 

/ I H/'lr  » 

and  from  Schwarz's  inequalitx  it  follows  that 


tot'd 


ir  Hi*,,  K H“* 

I .filing  - * ,*  ■ df  not  f l hf  standard  imiiw  prod  not  on  ( ’'  and  f xpanduvi  Kq  ( ;V7>,  we  got 


Q A,')’.)'  ' ' i'.  •/ 


,1.  o’  * « 


KV1'- 


t.ist 


NHL  RKPORT  8322 


where  the  matrix  K = K(t)  and  the  vector  J = J (v,  t)  are  defined  by 

K nm  ~^t  ’ K t ( /„ ),  (39a) 

rn  n m 

Jn=(e^K,  )=eAlJ-  <39b> 

n 

Kquations  (38)  and  (39)  arc  the  basic  equations  of  this  theory,  and  from  them  we  see 
that  the  error  function  Q2  is  a nonhomogeneous  polynomial  in  (I.  With  a little  calculus  it  is 
easy  to  show  that  the  “optimum”  value  of  (f  which  minimizes  Q-  is  given  by  $ = b,  where 
b = b (v,  t)  is  given  by 


b = K lJ.  (40) 

The  corresponding  minimum  value  of  Q-  is  denoted  by  Qf,  and  substituting  Kq.  (40)  into 
Kq.  ( 38)  we  get 

Ql  = lk„H2  - < K~l  J,  J > = ||et,||2  - <b,J>.  (41) 

When  II  = 1I(W)  the  weight  vectors  b are  the  tapered  weights  discussed  in  Section  2. 

We  shall  later  see  that  o-optimal  weights  used  in  calculation  of  /f^,  are  obtained  by  inverting 
the  matrix  K + 


5.2  The  Space  H(W) 

It  is  well  known  that  the  ordinary  Ip  norm  satisfies  Condition  1 on  the  class  of  band- 
limited  functions  f having  the  representation  ( 1 ),  and  for  a neat  account  of  the  IP  theory 
of  linear  operators,  we  refer  the  reader  to  Ref.  9.  However,  it  is  a standard  exercise  to  show 
that  the  IP  norm  does  not  satisfy  Condition  2,  and  we  therefore  define  an  inner  product 
(•,')  and  corresponding  norm  INI  by 

+ W _ 

(/.£)  = j • Dg(v)  di>,  \\f\\2  = (/-J),  (42) 

-W 

where  / ) denotes  the  differentiation  operator.  From  F,q.  (6)  we  have 

II /'ll 2 = A l2(f).  (43) 

Let  (T  (W)  denote  the  class  of  band-limited  functions  f = f(t)  whose  Fourier  trans- 
forms f = f(v)  are  of  class  C”  and  vanish  for  \v\>  W.  It  is  now  convenient  to  define  //(H  ) as 
the  completion  of  C”(H)  with  respect  to  the  norm  INI-  For  smooth  functions  f = f( c)  we 
have 
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lu*»r«\  from  Schwarz’s  inequality  vvr  have 


l/(<’,)  /(<’„  >1  2 • 1 1’,  I | I l>l  (i  )\  2 <//', 

/ 0 

and  thoivfoiv 

I /('„)!  \f  I *'|  r0l  || /'ll. 

From  this  inequality  it  is  easily  shown  that  the  norm  ||-||  satisfies  Conditions  1 ami  1!,  ami 
that  the  transforms/  ol  functions/  in  //(H  ) are  absolutely  continuous  and  vanish  at  the  cm! 
points  v - 1 H I he  transforms  f need  not  he  different  iablc  at  every  point,  and  if  the  trails 
forms  / , h which  appear  in  Kip  (12)  are  piecewise  smooth,  then  one  can  evaluate  the  integral 
by  integrating  on  each  piece  and  taking  the  sum.  More  generally,  one  can  replace  f,  g by  the 
appropriate  Cauchy  sequences  in  C((  ( H ) and  take  the  limit  as  n • «\ 

Remark  2.  Function  spaces  whose  norms  incorporate  IV  norms  of  /,- 1 h order  derivatives  are 
called  Sobolev  spaces,  and  are  generally  denoted  by  the  symbols  /.{!,  H k , and  II  k for  the 
case/i  2.  They  are  currently  much  in  use  in  differential  equations  and  m the  calculus  of 
variations,  and  we  refer  the  reader  to  Kefs.  10,  11,  and  12  for  the  linear  theory  and  to 
lief.  1 2 for  the  nonlinear  theory.  Condition  2 for  the  norm  of  //( H ) is  a particular  example 
of  Sobolev  Embedding  Theorem.  Sobolev  space  techniques  have  also  been  used  by  Sobolev 
and  his  students  in  studies  of  efficient  numerical  quadrature  (eubature)  on  high-dimensional 
manifolds,  and  we  refer  t he  reader  to  the  expository  paper  by  I labor  | 1 1 1 for  an  account 
of  this  subject. 


5.;$  Basic  Formulas  For  II(VV) 

When  //  //(  H),  the  functions  C«> “ (/-(r,  /,  p’)  can  be  computed  for  general  i>,  /,  fl  by 

means  of  Kips.  (.’18),  (39),  and  the  following  identities: 


K,(t) 

1 

\ sin | 2rrVV( / s)|  sin(2irHs)sin(2;rH7) 

4*3 

) 8t(t  s)  2illsV’ 

K,(t) 

1 

\ sin”(2trH7)  1 

, ) 2irWt  . (/  / 0). 

1 1 2trW7  \ 

4 r]r 

K„(M 

K,|0) 

1 ( 

..  xsin(2ffH7)  (2/rH7)cos(2TrH7) 

4r*r  t 

<> 

K0(  0)  “ W \ 


( I la) 


( Hb) 


(I  1c) 


( I Id) 


II 


NUI  Kt'.PUKT  H:li2 


*Y<'> 

•>  •>  | I'-'"1  ’ eos(2irW7  1 

4ir  / t 

(u  W )sin(2nW7lJ  , ((  / 0),  , .|f,a) 

«>» 

(1/2)  (If2  < -), 

(45b) 

II 

1 *v  ii 2 

(HJ  r-)/(2W) 

( 45e) 

data  art'  sampled  at  the  Nyquist  rate  2W  . /„ 

n/(2H  ),  anti  Kqs  |II9)  speeiah/.f  to 

Kmn 

0. 

(in  > n,  m * 0,  n t 0), 

^ m m 

(2H'  t )/(n~w  ’ ), 

(m  * t)), 

(46) 

^ mo 

( 1 1"1  * 1 (2W  1 )/( ir'm  2 ), 

(m  * 0), 

. K"" 

(2/:i|H;i. 

r J 

m 

| H “/(n"ni-)|  |exp(irtmr  H I 

( 1 >"*  I.  (m  / 0). 

1 

J 

it 

(1/2)  (IV-  e-). 

(47) 

Ik,, II2  ( H’~  i -)/(2M  > 

l\i  math  .1  I he  baste  inequalities  (7)  and  (8)  art'  diroot  ct>nst't|Ut'ii('t's  of  Kqs.  (1  Hi)  ami 
(4f>o).  Kor,  iisinn  thf  Soli  war/  int>t|iiality  wr  not 

IAMI*  \U.KI)\~-  II/'  ||-  HA’,112  ||/-||2  K,(M. 

l/(«’)l"  K/>,  ) I"  • ll/'ll2  Ik-,,112. 

I'roofs.  Intt'nratmn  Kq.  (d  ll  by  |>arls  wr  obtain 

+ W 

/(O  (f.Kt)  J fir)  • n2Kti |)  dr 

w 

llt'in-f,  fomparmt-  this  equation  with  Kq.  ( 1 ) wr  stf  that  Kt  tf,(i)  is  tin-  solution  to 

| t)  * . \ 

\ n~Kt(r)  ■=  o\p|  2tnrf|;  K{  ( IV)  K,(H’)  ().' 


If. 


. 
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(The  boundary  conditions  K,(  * W)  0 are  required  because  K,  belongs  to  the  class  //(  M), 
whose  transforms  vanish  at  the  endpoints  v = • W.)  Hence,  for  t * 0 we  have 


K,(f)  = 

•>  0 ! •'  llUi 1 ‘‘os(2jrW()  + (iV/R')sin(2nW7)^  , 

4ff-r 

v I 

(49) 

K0(r)  = 

(l/2)(  W - e2). 

To  obtain  Kqs.  (44),  compute  Ks(t)  = (K#,  ) directly  from  Kqs.  (42)  and  (49). 

To  obtain  Kqs.  (45),  we  have 

= (V  Kt)  = (Kret,)  = K((r), 

and  we  use  Kq.  (49).  It  is  interesting  to  note  that  e,.()(e)  is  piecewise  smooth,  with 

‘Y  O')  = 1 1/(2W')|  }( H - i>ft)  (v  - »’„)(tV  h(v  - vg)  + r())[,  (50) 

where  h is  the  Heaviside  function,  h(r)  = sign  (r). 

5.4  Derivation  of  Formulas  for  tj" 

Let  the  data-point  set  Jt„  j he  given  by  tn  = nj(2W)  where  n varies  over  the  set  of 
A/,  = ( 2N  + 1)  integers  jn : ,V  n ^.JV| . The  conventional  weights  are  given  by  (5„  = c„ 
where 


cf|  = (1/2W  ) exp  (ninv/W), 

and  by  direct  substitution  into  Kqs.  (1)8),  (46),  (47),  and  (48)  we  get 

t N t 

Q'f.  = (W72)|(l/3)  - (2/tt2)^  (l/n2)>. 

1 

We  shall  have  occasion  to  use  the  identities 


(51) 


(52) 


E 


cos  2jrnt  0 1 

= 7T“(f2  t~  -),  0<(<1. 


(53a) 


+oo 


E 


exp  2nint  n exp  2irt/y  + exp  2tr(l  t)/y 


1 + 7“  n 


2 „2 


exp  2n/y  - 1 


(53b) 
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whose  proofs  consist  in  verifying  that  the  left-hand  sides  are  the  Fourier  series  of  the  right- 
hand  sides.  Setting  t = 0 in  Eq.  (53a)  we  get  the  well  known  identity  S(l/n2)  = 1/6.  Hence, 


<?2  = (W7tt2)  £ (1  In2), 

N+ 1 


and  Eq.  ( 16)  is  obtained  using  the  approximation 


£ 

N+ 1 


(1/n2)  s: 


1 

N+l 


N 


5.5  Derivation  of  Formulas  for 

We  shall  now  sketch  the  calculation  of  the  o-optimal  weights  defined  in  Section  3.3. 
These  are  the  weights  that  minimize  R2  for  given  values  of  Af2  and  a2,  and  referring  to 
Eq.  ( 14)  we  see  that  the  tapered  weights  (defined  in  Section  2.3)  are  obtained  by  setting 
a = 0.  We  shall  assume  tn  = n/(2W)  where  n varies  over  a set  L of  integers,  and  that  the 
integer  0 belongs  to  this  set.  (However,  the  set  is  not  assumed  to  be  symmetric  about  0.) 

Referring  to  Eqs.  (14)  and  (38),  we  have 


r2/a/s 


= <<*+  ~ <P,J>  - <jJ>  + ||e„||2. 


Hence  the  minimum  occurs  when  (A'  + (<j2/A/2)  /)  (f=  f,  and  from  Eq.  (46)  this  equation 
reduces  to  the  system 

2 

Kmo$o  + (Kmm  + > Pm  = J„r  * °>, 


<Koo  + + = Jo' 


where  1/  denotes  summation  over  the  nonzero  integers  n in  L.  The  solution  is  readily 
obtained  by  solving  first  for  j3„,  and  then  for  (30 . Let  the  quantities  a,  a,  A . /?  C be  defined 
by 


a = vjW, 


(54) 


a = 


2 rAl, 


1/2 


2/(  7T  AM, 
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(A 

i(')  (211  H), 

1 

' y*  C*  ♦ 
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t 1 tP , ' I. 

y"  2H 

an,l  ll',‘  corresponding  minimum  \ .ilu<>  of  l{~  is  pneii  l,\ 


«0  “ 


(55) 


( ot> ) 


l-  M-  ( 

■m  \ 

Hu-  value  of  Hi  IS  computed  mvonlmu  to  !.,  (122).  and  therefore  each  of  the  sums  m 
K,,s  (.,5»  becomes  an  infinite  senes  1)  i„.  „ ♦ Ihen  C 0.  and  each  of  the 

mfmil.-  scries  ...  the  expressions  for  I .uni  li  is  decomposed  ...to  tin-  sum  of  tsxo  infinite 
scrn-s  by  means  ol  tin-  idi-ntil\ 


toT) 


n-’(  1 ♦ a W-)  n~  i ♦ 

Hosed  form  expressions  for  tin-  resulting  s,-rn-s  an-  then  obtained  by  means  of  K«t  (5dh) 

As  already  mentioned,  expressions  for  tin-  taper.-, 1 weights  are  obtained  from  t o ,56) 
by  set  inn  a l)inh|  ( 55),  and  the  riKht  hand  side  of  K,|  (57)  then  represents  It  Q-, 


Finally,  in  the  d< 
this  is  not  the  ease.  K 
we  fjet 


■rival ions  we  have  assumed  that  tin-  data  point  set  contains  zero  When 
is  a diagonal  matrix  and  the  derivations  an  even  simpler  In  particular. 
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